In projecting images using an optical projector having one or more spatial light modulators, various lamps, lenses, reflectors and spatial light modulators such as digital micromirror devices (DMD), liquid crystal display (LCD) and liquid-crystal-on-silicon (LCoS) devices are often used. The optical projection systems are typically formed in two parts: an illumination system for generating and collecting the light rays needed to illuminate an image, and a projection system for collecting the illuminated image light rays into a projection lens and then projecting the final image out of the compact optical device. At least one spatial light modulator device receives the illumination light rays from the illumination system, modulates the light received with image data for visual display, and transmits image light rays for projection to the projection system. In many systems the spatial light modulator is an array of reflective pixel elements such as mirrors that receive digital image data and reflect or don't reflect the illumination light corresponding to the digital image data.
There are several challenges associated with providing an optical projection feature in a battery powered device such as a portable projector, tablet, laptop or smartphone using prior known solutions. Because the device often operates on battery power, there is a continuing need to reduce the power consumed to extend battery life between charges. Brightness is often also a consideration. The projector brightness efficiency performance metric, which can be for example lumens/Watt, must be optimized.
In order to further illustrate the operation of the prior known solutions, FIG. 1 depicts in a system block diagram a top view or plan view of a prior known projection system 40. System 40 is configured using conventional reverse total internal reflection (RTIR) projection architecture. In an RTIR architecture, a reverse total internal reflection (RTIR) prism is used in a projection path between a spatial light modulator and the projection optics, as is further described below. However, the arrangements of the present application are not limited to a particular illumination system or projection system, and FIG. 1 is presented as a non-limiting example of a typical projection system for illustration only.
In the projection system 40, illumination is provided as shown as provided by the use of red, green and blue (RGB) LEDs 42, 43 and 46. However, alternative illumination sources can also be used, such as incandescent lamps with reflectors, single lamps with color wheels, laser, laser-phosphor illumination, and the like. The LEDs can include an optical coating or collimating optics 41 which act to collect and collimate the light output by the LEDs. Also, as illustrated in FIG. 1, two LEDs 42 and 46 are shown on a single integrated device, these can be the red and green LED devices, for example, while the blue LED 43 is provided as a separate component. In alternative systems three individual LEDs are used, and two dichroic plates in the form of an X box shape can be used to combine the three colors (RGB) into an illumination source. In the particular example shown in FIG. 1, dichroic plate 48 reflects the light from red LED 46 at one surface, reflects the light from green LED 42 at a second surface, and passes the light from blue LED 43 through and to the illumination path. Note that in alternative arrangements, many LEDs can be used or multiple LEDs can be used instead of one LED for each color.
In FIG. 1, an additional collimator 49 for example is placed between the LEDs 42, 46 and the dichroic plate 48. Collimators are well known and perform the function of reducing the beam diverging angle. Integrator 47 is placed in the illumination path after the dichroic plate. The integrator 47 may be a “flys-eye” integrator (also referred to as a lens array), or a rod integrator or tube integrator. The integrator 47 produces a more homogeneous light beam which can then be transmitted through one or more relay lenses such as relay 51. The relay optics such as 51 can extend the length of the illumination path.
Mirror 61 is provided and in this particular example arrangement, folds the illumination light path. This reflective fold mirror also enables the illumination light rays to reach the spatial light modulator 73, which in this example can be a digital micro-mirror device, at an angle. Because the digital micro-mirror (DMD) 73 modulates the light by tilting reflective mirrors, the illumination rays must strike the mirrors at an angle. Use of the folding mirror 61 makes control of the angle of the illumination rays to the spatial light modulator 73 easier to achieve. Additional relay optics such as 52 can be placed between the mirror 61 and the DMD 73.
Use of a reflective spatial light modulator such as DMD 73 requires that the illumination light rays from mirror 61 that are entering the DMD package and the reflected image light rays leaving the DMD mirrors in spatial light modulator 73 be physically separated to avoid interference, as can be seen by examining FIG. 1. As is known in the art, the use of an RTIR prism can separate the incoming rays from the illumination system from the image rays that are being transmitted into the projection optics. U.S. Pat. No. 5,309,188, entitled “Coupling Prism Assembly and Projection System Using Same,” which is hereby incorporated by reference in its entirety herein, discloses a prism arrangement using total internal reflection to separate the illumination and projection light paths in a small space. As shown in FIG. 1, wedge prism 75 and TIR prism 76 form a coupling prism that accomplishes the needed separation of the illumination light rays directed onto the spatial light modulator from the image light rays coming from the spatial light modulator. The image light rays exit prism 76 and are coupled into a projection system that includes elements 54, 56, and 59
FIG. 2 depicts in a simple circuit block diagram a typical arrangement 200 for use with the compact optical projection system described above. A microprocessor, mixed signal processor, digital signal processor, microcontroller or other programmable device 211 is provided and executes instructions that cause it to output digital video signals for display. A variety of sources may provide the digital video signals labeled DVI in the figure, including television transmitters, cable boxes, internet browsers, stored files in video cards, flash cards, USB drives and the like, cameras, computers and camcorders, etc. The microprocessor 211 is coupled to a digital DMD controller circuit 203. DMD controller 203 is another digital video processing integrated circuit. Sometimes this controller 203 can be implemented using a customized integrated circuit or an application specific integrated circuit (ASIC). An analog circuit configured to manage power and LED illumination referred to as a power management integrated circuit (PMIC) and numbered 215 is also provided. The PMIC 215 controls the intensity and power to the LEDs 209. The DMD controller 203 provides digital data to the DMD 201 for modulating the illumination light that strikes the DMD surface, and the PMIC DMD controller 215 also provides power and analog signals to the DMD 201. The light rays from the illumination sources LEDs 209 are input to illumination components in block 215 such as the cover prism and wedge described above, and strike the reflective mirrors inside the package of DMD 201. The reflected light for projection leaves the surface of the DMD 201 and travels into the projection optics 207 which operate to project the image as described above. Together the integrated circuits 203, 215 cause the DMD 201 and the optical components 215, 207 to operate to project the digital video signals as an image.
Example integrated circuits that can be used in the circuit shown in FIG. 2 include DMD controller ICs from Texas Instruments Incorporated. The DMD controller ICs that can be used include, for example, the DLPC3430 DMD controller, and the DLPC2601 ASIC device that can provide both digital and analog controller functions. Analog DMD controller devices from Texas Instruments, Incorporated that can be used include the DLPA2000 device. LED controller devices can be used to power on and off the RGB LEDs, for example.
The spatial light modulators can be implemented with DMD devices from Texas Instruments, Incorporated such as the DLP2010DMD, which is a 0.2 inch diagonal device that provides wide VGA (WVGA) resolution. Many other DMD devices are available from Texas Instruments Incorporated that can be used in digital projection systems.
In FIG. 2, a high speed interface labeled “I/F” is used to couple signals from the DMD controller or ASIC 203 to the DMD device 201. In a prior known approach, a complete set of pixel data is transmitted to the DMD many times per second, for example if the incoming data frame rate is 60 Hz, the interface receives pixel image data for three different colors (red, green, blue, for example) 60 times per second. In the prior known approaches, there is no processing used to compress or reduce the data rate on the interface I/F between the DMD ASIC 203 and the DMD 201. As the devices are typically implemented in CMOS technology, the power consumed is directly proportional to the switching speed. Thus as the pixel data is transmitted on this high speed interface, the driving devices switch on and off and consume power. This power is consumed even when the system is projecting a static image. For example, during a power point presentation, the image projected can remain static for many frame cycles. The constant transmission of pixel data from the DMD ASIC 203 to the DMD 201 causes substantial power consumption with no additional benefit, as the image viewed at the screen (in cases where the image displayed does not change) remains the same over many frame periods.
In order to further explain the arrangements of the present application, additional concepts related to the use of pulse width modulation as applied to the display of images using a spatial light modulator, such as a DMD, are now presented.
The use of digital data for image display is convenient for processing and storage, and projecting data from a frame buffer at discrete time periods is very practical when using a pixel array formed of addressable elements that “latch” the current value. However the human visual system (HVS) is an analog system and so, if unwanted visible artifacts that appear to the viewer due to the use of digital image data are to be avoided, certain steps are necessary to break up the images into smaller portions that update frequently.
A “frame display time” can be determined using the typical metric of “60 frames per second” as the minimum display rate needed for the human visual system to see continuous motion in an image displayed for viewing. However, particularly when color is used and some portions of the image have multiple colors, faster frame rates are known to reduce or avoid “rainbows” and other visible artifacts that can sometimes be observed. For example, a 2× rate of 120 frames per second may be used.
Further “bit planes” or “sub-fields” can be defined to format the images for the spatial light modulator and also to further improve the images for display to the HVS. Because the pixel elements for a binary spatial light modulator are either “on” or “off”, the intensity observed for a particular pixel is determined by the amount of time that pixel is on during the frame display time. The image data coming into the ASIC device may have several bits for each pixel, for each of the colors, to represent color intensity for a particular frame. A binary spatial light modulator can only process one bit per pixel for each image, so a mapping is performed to create the intensity levels needed for each pixel during the frame display time at the spatial light modulator. By subdividing the frame display time into bit planes, each having a bit for each of the pixels in the two dimensional array at the spatial light modulator, a variety of intensities, corresponding to a “gray scale” for one color, can be achieved. If the pixel is “on” for the entire display time, it will have a maximum brightness or intensity. If the pixel is “off” for the entire time, it will be dark, or have a minimum brightness or intensity. By using the bit planes, the entire range of color intensity available can be reproduced using the one bit per pixel available in the spatial light modulator.
Further, the bit planes may have non-uniform display times associated with them. In this manner a weighting function is implemented. A first bit plane may be displayed for half the frame time, a second for one quarter, a third for a smaller portion, and a fourth for a still smaller time portion, for example.
In addition, to further adapt the display of the bit planes to the workings of the human visual system, the bit plane display times can be broken up so that, in the above example, the first bit plane is displayed twice in a rotation, for two different times, thus the visual image being displayed updates more frequently. The HVS will see fewer artifacts with additional breaks in the image being projected. Because the time each bit plane is displayed at the DMD is non-uniform, the pulses transmitted to the DMD array are said to be pulse width modulated, the width of the pulses corresponding to the relative weight given a particular bit plane. The pulse width modulation therefore controls the intensity for the pixels.
In a further aspect of the prior known image display projection systems using binary spatial light modulators, these SLMs have only one bit per pixel of image storage. In use, the picture elements, or pixels, contain a latch and a reflective device associated with the latch, such as a mirror. In conventional devices there is a one bit storage cell associated with the pixel as well. While a particular pixel frame is being displayed by the pixel, a single bit memory cell located adjacent to the pixel is loaded with the next pixel value for display. When the line or the array of pixel data is loaded, a “reset” will cause all of the pixel elements to update (latch) the next value for display, and the one bit storage loading process can begin again.
Because the SLM has little internal storage, the DMD controller circuit necessarily transmits multiple bit planes to the SLM for each pixel during each frame display time period, and these bit planes often contain redundant or repeated data.
FIG. 3 illustrates a system 300 and portions of the data path for a conventional digital imaging system using a DMD controller ASIC 301 and a SLM device 303, for further explanation.
In FIG. 3, the DMD controller 301 receives pixel data. A block 311 converts the incoming pixel data, which has multiple bits for each color for each pixel, these are mapped into images referred to as bit planes. In an example, a frame for display can include colors red, green and blue and can include, in one example, 8 bits per pixel for each color or 24 bits for each pixel. The conversion block 311 converts the image frame data received by the ASIC into bit planes and stores the bit planes in a frame buffer 305. In addition, the data can be formatted prior to writing the data to the frame buffer 305. In a second frame buffer 307, the data is read from the frame buffer 305. By switching between the two frame buffers, the frame being read for transmission to the SLM is separated from the frame being written with bit planes corresponding to the incoming frame image data, enabling the system to operate continuously, receiving data, converting and writing bit planes to a first frame buffer, while simultaneously reading bit plane data from a previously loaded frame buffer. After the data is read from the frame buffer such as 307 in FIG. 3, additional data formatting can be performed to ready the data for transmission on the high speed interface I/F. For example, in a packet data interface, the data packets can be formed.
In order to ready the image data for display for viewing by the HVS, as explained above, a plurality of bit planes are displayed during each frame display time period. The bit planes can include three colors for each pixel and are arranged so as to create the desired pixel intensity. In order to break up the changes in the image to provide better displayed image quality with few or no visible artifacts, the number of bit planes transmitted to the SLM is increased from 24 (8 bits per pixel for 3 colors) to 60 or more, with some repeating. In a typical application, some of the bit planes are repeated, and since the known prior solutions do not make any correlation between the data transmitted for the various ones of the bit planes, all of the pixel data for each bit plane is transmitted over the data interface I/F in FIG. 3 to the SLM device 303. Blocks 311 and 315 in the controller circuit 301 provide formatting that creates the bit planes from the read frame buffer and then transmits these bit planes to the SLM 303 on the data interface labeled I/F.
As shown in FIG. 3, the SLM 303 includes high speed logic 317 and a data receiver 319. The data transmitted from the DMD controller 301 is received by the high speed logic 317 which writes the storage cells for the mirror array 319, and when the entire array, a line, or another portion is ready, a “reset” to the DMD pixels causes them to update with the data from the storage cells and display the new image corresponding to the bit plane. The load and reset process is repeated for all of the bit planes for a particular image display time to provide the color and intensity information for viewing by the HVS. The image is displayed by illuminating the DMD mirrors and the reflected images are projected for viewing at screen 309 for example. The display is reset each time a bit plane is loaded so that all of the bit planes for a given frame are displayed in a sequence during the frame display time.
In an example with an image display having four bit planes, referred to as bit planes 0-3 for a given color that are to be displayed during a predetermined color display time, each bit plane has an associated time weighting. For a four bit plane example, the weights might be 1, 2, 4, and 8. Base on the sum of the time weights, in this example the sum is 15, the associated time proportion for bit planes 0, 1, 2, 3 might be 1/15, 2/15/, 4/15, 8/15. When a particular bit plane is displayed with multiple instances, its total display time will still match the corresponding display proportion. When a bit plane is displayed with more than one instance in a display time, the bit plane is referred to as being “split”. For example, in a bit plane sequence of ‘30213’ bit plane 3 is a split bit plane.
In a typical bit splitting approach, when the prior known solutions are used, the same pixel in the SLM is re-loaded multiple times during the color portion of the frame display time. This means the data corresponding to that pixel are all transmitted from the DMD controller 301 to the spatial light modulator device including the DMD over the high speed interface I/F for each of the bit planes, in this example, 5 times (a sequence of 30123). If the binary data within each of these bit planes is independent, then the entire 2D bit plane must be transmitted for each of the 5 bit planes. Since there is no correlation between the data contained within each bit plane, all of the binary states must be transmitted for each bit plane instance. A system having correlation between bit planes could result in the decreased transmission, resulting in power reduction.
Improvements in the power consumption for optical projection systems are therefore needed in order to address the deficiencies and the disadvantages of the prior known approaches and to further improve performance. Solutions are needed that reduce the power consumed, reduce the data transmitted on the interface between the DMD controller and the spatial light modulator, and which extend the battery life for portable devices, while overall system performance is maintained or improved.